The present invention relates generally to optical signal processing systems, and in particular to systems that extract features from optical images for pattern recognition. Pattern recognition using imaging sensors can be implemented by means of either feature extraction or image matching. Optical image correlators have been developed, for example, for updating inertial navigation systems, and optical image matching systems have also been developed for pattern recognition. Many different types of optical feature extractors for pattern recognition have also been investigated. Generally speaking, optical processing as described herein offers a fast and highly parallel method of feature extraction and correlation using the fundamental properties of wavefront multiplication, addition, rotation, splitting, and feedback. On the other hand, one of the key concerns in the design of optical feature extractors is that the features selected for pattern recognition should be invariant with respect to scale and rotation, which simple image correlators are particularly sensitive to in most cases. As will be described in more detail hereinafter, such sensitivity to rotation and scale can be obviated by using video feedback.
Despite the fundamental advantages of optics for processing images (optical Fourier transforms, high parallelism and interconnectivity), one of its technological disadvantages stems from the lack of good two-dimensional (2-D) spatial light modulators. As a result, optical processing has been restricted to a few applications where 2-D operations can be broken down into sequential one dimensional (1-D) operations (e.g. synthetic aperture radar), for which well-developed (1-D) acousto-optics and television (TV) raster signals are used. See, for example, D. Psaltis, "Acousto-Optic Processing of Two-Dimensional Signals", J. Opt. Soc. Am. 71 @ p. 198 (1981). Another strategy is to exploit the mathematical properties inherent in the Radon transform and, as will be shown, angular correlation. These operations are particularly useful when viewing objects in plan view that possess some degree of rotational symmetry and scenes with good signal-to-clutter ratio. A helpful treatise on the Radon transform is contained in S. R. Dean, "The Radon Transform", in Mathematical Analysis of Physical Systems, R. E. Mickens, ed., Van Nostrand Reinhold Co., N.Y., pp. 81-133 (1985). (Previous treatments of angular correlation are not known.)
Image forming systems such as synthetic aperture radar (SAR) typically form images in plan view within time spans on the order of one second, thus allowing frame-to-frame algorithms to be implemented at video frame rates. After the SAR image is formed, features can be extracted using video feedback at TV rates to support pattern recognition within the required SAR image formation time. If faster decision rates are required, as would be the case for TV-based parts inspection or TV-compatible infrared or millimeter-wave surveillance, multi-aperture micro-optical components can be employed to support feature extraction and pattern recognition (via the Radon transform and angular correlation) without video feedback. In either approach, one of the major requirements on the sensor is that it automatically track the desired object so that shift invariance is virtually insured.
Prior experience with digital image processing of SAR imagery of ships has shown that longitudinal range profile (i.e., Radon transform) -based classification is useful, but segmentation is required before feature extraction. Both are time consuming and complex. Extracting longitudinal range profiles (Radon slices), which are virtually the optimal 1-D information of ships, from 2-D SAR imagery is particularly attractive because the Fourier transform of range profiles is a well developed and useful technique for ship recognition. As will be described in detail hereinafter, the optical Radon transform architecture proposed in accordance with the present invention determines the Radon transform naturally using simple optics, and the selection of the desired (longitudinal) profile is enabled by the use of an angular correlation architecture, which obviates image segmentation. In such application, a liquid crystal television (LCTV) is used in a transmissive shadow casting mode for displaying a reference image in the angular correlation architecture. The Fourier transform of the resulting profile (Radon transform) can then be computed after selection by the angular correlator.
Optical realizations of the Radon transform have been implemented by a number of investigators. Both forward and inverse Radon transforms have been considered, inverse transforms being used for tomographic reconstruction. However, all of these techniques involve the use of rotating or translating objects, lenses, drums, slits or film. One of the simplest architectures used to create a Radon transform uses a mechanically driven image rotator with anamorphic optics and a 1-D output detector array to obtain successive slices of the Radon transform. See G. R. Gindi and A. F. Gmitro, "Optical Feature Extraction via the Radon Transform", Optical Eng. 23 @ p. 499 (1984). The image rotator could be a dove prism or fiberoptic (bundle) coupler. The problem with these mechanical rotation schemes is that they are bulky and generally less reliable than all-electronic/optical schemes. As will be described later, the use of video feedback in accordance with the present invention eliminates the need for mechanically rotating components.
In pattern recognition, a key processing step after sensing the information is feature extraction, and the Radon transform is an effective method for feature extraction using different types of sensors. Optical parts inspection is one application using television sensors; see, for example, M. J. Simpson, P. A. Ervin and M. A. Snyder, "Radon Transform Applications in Optical Inspection", Optical Eng. 27 @ p. 164 (1988). Other sensors of interest include SAR, infrared, and millimeter wave sensors for surveillance and target recognition (especially when they display plan views of the desired objects).
In the case of objects with a high degree of bilateral symmetry, a longitudinal slice of the object intensity distribution, at a particular orientation, yields the most structural information. To obtain this result digitally requires an algorithm that estimates boundaries of the object via segmentation, determines the orientation of the object (longitudinal axis) in the scene, collapses the object's intensity samples perpendicular to the longitudinal axis, and then samples the resulting profile along the longitudinal axis. Such an algorithm is complicated to implement digitally and is subject to errors in estimating object scale (e.g.: length) and orientation. Once the longitudinal profile is found, however, it can then be processed by Fourier transformation, etc. to obtain suitable features for classification. For objects with higher degrees of rotational symmetry, more Radon slices may be required for classification of the object. Thus, an important prerequisite for deriving the longitudinal Radon slice is determining the orientation and symmetry of the object.
A way in which the object orientation and symmetry can be decided, for a given input scene containing a long or symmetric object, is to perform angular correlation. This will allow determination of the Radon transform without detailed knowledge of the object boundaries via segmentation. Thus, by rotating a reference image with respect to the sensed image, the correlation will be maximized when the images are aligned at the angle (.phi.), relative to the reference image axis of symmetry, corresponding to the longitudinal axis of the object. When this occurs, the Radon transform of the input (sensed) image should be taken. Recovery of the orientation angle, i.e., the angle required to bring the sensed image into alignment with the reference image is an important parameter in sensor vehicle guidance.
Normally, correlation for scene matching or target tracking is implemented by shifts in cartesian coordinates (x,y). Rotation in angle (.phi.) about the optical axis, or shifts in scale are undesirable, since the correlation peak-to-sidelobe level would be degraded. Much prior work in optical image matching system development has been devoted to eliminating these scale and rotation errors or making processing schemes that are invariant with respect to them. See for example, U.S. Pat. No. 4,084,255 issued to Casasent and Psaltis. In contradistinction, an angular correlator uses angle shifts as the lag variable and is sensitive to cartesian coordinate offsets. It is not sensitive to scale because angular correlation can be performed without a separate reference image, i.e. it is an autocorrelation that can be normalized by the peak correlation value. A SAR system, however, is designed to generate images which should not display errors in scale by the very nature of the radar signal processing, and that should be shift invariant within limits set by slant-range and cross-range (Doppler) tracking accuracy. Optical, infrared or millimeter-wave image sensors also can be made to generate images under the control of a tracking system so that shift errors are minimized. Scale sensitivity again can be eliminated with angular correlation, as just described.
Optical feedback has been used to perform a number of functions including pattern recognition, temporal and nonlinear processing, iterative transformation algorithms, and all-optical numerical computing. Television-based processing, with its ability to handle 2-D signals, can be utilized in optical architectures involving feedback. Thus, Crutchfield in "Space-Time Dynamics in video Feedback", Physica 10D @ p. 229 (1984), has studied the properties of video feedback, characterizing it as a chaotic system under some conditions. In contrast, the use of video feedback as proposed in accordance with the present invention and as described hereinafter aims to eliminate these dynamic instabilities. This is accomplished by the simple but crucial expedient of uncoupling successive video frames, using an input video gate and feedback video frame delay.
A number of investigators have also studied the application of feedback in optical information processing. See, for example, "The Use of Feedback in Optical Information Processing" by J. Cederquist and S. H. Lee, Appl. Phys. 18 @ p. 311 (1979). Moreover, earlier research has demonstrated the feasibility of using video feedback, by means of a beamsplitter, to enable readout of the iterated images in the feedback loop, see G. Hausler and A. Lohmann, "Hybrid Image Processing with Feedback", Opt. Commun. 21 @ p. 365 (1977), while still other architectures have been investigated that were used for affine image transformations, see E. S. Nezhevenka and B. I Spektor, "Affine Pattern Transformation in Optical Systems with Feedback", Avtometriya, No. 6, pp. 14-18 (1976). In addition, as taught by E. Marom et al in "Pixel-by-Pixel Array Division by Optical Computing", Optics Lett. 10 @ p. 43 (1985), liquid crystal light valves have also been used in lieu of television systems in incoherent feedback systems. Finally, recent work has also been reported by A. E. T. Chiou and P. Yeh in "Scaling and Rotation of Optical Images Using A Ring Cavity", Appl. Optics 29 @ p. 1584 (1990) as relates to the scaling and rotation of optical images in a laser ring cavity. However, none of these investigations have reported any attempt to separate (i.e. gate and delay) successive video frames or to process outputs of the video feedback loop as proposed in accordance with the present invention, and as will be described in detail hereinafter.